Method and apparatus for segmenting a two-dimensional image of an anatomical structure

ABSTRACT

There is provided a method and apparatus for segmenting a two-dimensional image of an anatomical structure. A three-dimensional model of the anatomical structure is acquired ( 202 ). The three-dimensional model comprises a plurality of segments. The acquired three-dimensional model is adapted to align the acquired three-dimensional model with the two-dimensional image ( 204 ). The two-dimensional image is segmented by the plurality of segments of the adapted three-dimensional model.

FIELD OF THE INVENTION

The invention relates to the field of medical imaging and, inparticular, to a method and apparatus for segmenting a two-dimensionalimage of an anatomical structure.

BACKGROUND OF THE INVENTION

Medical imaging is a useful tool for providing visual representations ofanatomical structures (for example, organs) in images. There exist manydifferent types of medical imaging techniques including computedtomography (CT), magnetic resonance (MR), ultrasound (US), and similar.The images acquired from medical imaging can prove valuable for clinicalanalysis of a subject and, where necessary, medical intervention.

In many clinical applications, the accurate segmentation oftwo-dimensional (2D) images as well as three-dimensional (3D) images isrequired. Three-dimensional, or volumetric, images usually cover thewhole object of interest (which may, for example, be an anatomicalstructure or part of an anatomical structure). The segmentation ofthree-dimensional images can benefit from context information in allthree spatial directions. Moreover, the topology of the object in athree-dimensional image is consistent. On the other hand,two-dimensional images are often superior in terms of signal-to-noiseratio and spatial and/or temporal resolution. However, while algorithmshave been proposed for the segmentation of multiple slices with knownspatial relations, these algorithms cannot be employed for a singletwo-dimensional image. Also, the algorithms do not benefit from modelslearned from three-dimensional data.

Some existing techniques for segmenting two-dimensional images involvelearning two-dimensional shapes from slice images. However, thesetechniques are impacted by inconsistencies in the placement of the scanplane during image acquisition. Also, depending on the exact scan plane,the topology of the two-dimensional contours may change. Moreover, theamount of training data available from two-dimensional scans is limitedsince only model points that lie on the slice of the training image areaffected for each training instance.

WO 2016/110463 discloses a method in which two-dimensional image data ofan object is segmented by applying a two-dimensional model to thetwo-dimensional image data, where the two-dimensional model isdetermined from a three-dimensional model. However, in this approach,the derived model cannot compensate for variances or inaccuracies in thechoice of scan plane for the two-dimensional image data. Moreover, thevolumetric information is lost in the process of creating thetwo-dimensional model and thus can no longer be used for visualisationor computation of certain parameters (such as the volume of certainanatomical structures).

Therefore, the existing techniques for segmenting two-dimensional imagesare susceptible to inaccuracies and only limited information can beacquired from two-dimensional images segmented using the existingtechniques.

There is thus a need for an improved method and apparatus for segmentinga two-dimensional image of an anatomical structure.

SUMMARY OF THE INVENTION

As noted above, the limitations with existing approaches for segmentingtwo-dimensional images are that the techniques used are susceptible toinaccuracies and only limited information can be acquired fromtwo-dimensional images segmented using these techniques. It would thusbe valuable to have a method and apparatus that can segment atwo-dimensional image of an anatomical structure to overcome theseexisting problems.

Therefore, according to a first aspect of the invention, there isprovided a method for segmenting a two-dimensional image of ananatomical structure. The method comprises acquiring a three-dimensionalmodel of the anatomical structure, the three-dimensional modelcomprising a plurality of segments, and adapting the acquiredthree-dimensional model to align the acquired three-dimensional modelwith the two-dimensional image. The two-dimensional image is segmentedby the plurality of segments of the adapted three-dimensional model.

In some embodiments, the acquired three-dimensional model may comprise aview plane associated with the two-dimensional image. In someembodiments, the view plane associated with the two-dimensional imagemay comprise a two-dimensional plane through the anatomical structure.In some embodiments, the two-dimensional plane through the anatomicalstructure may be defined with respect to one or more anatomical featuresassociated with the anatomical structure. In some embodiments, theacquired three-dimensional model may be adapted to align with thetwo-dimensional image based on the view plane associated with thetwo-dimensional image. In some embodiments, the acquiredthree-dimensional model may be adapted to align with the two-dimensionalimage based on any one or more of a spatial position of the view planeand an orientation of the view plane.

In some embodiments, the acquired three-dimensional model may compriseinformation associated with one or more of the plurality of segments,the information corresponding to one or more characteristic features ofthe anatomical structure. In some embodiments, the acquiredthree-dimensional model may be adapted to align with the two-dimensionalimage based on the information corresponding to the one or morecharacteristic features of the anatomical structure.

In some embodiments, adapting the acquired three-dimensional model toalign with the two-dimensional image may comprise any one or more ofrotating the acquired three-dimensional model to align the acquiredthree-dimensional model with the two-dimensional image, and translatingthe acquired three-dimensional model to align the acquiredthree-dimensional model with the two-dimensional image. In someembodiments, adapting the acquired three-dimensional model to align withthe two-dimensional image may comprise restricting the degrees offreedom of the acquired three-dimensional model.

In some embodiments, adapting the acquired three-dimensional model maycomprise minimising an energy functional that attracts thethree-dimensional model to the two-dimensional image to align thethree-dimensional model with the two-dimensional image. In someembodiments, the energy functional may comprise any one or more of aninternal energy term that constrains a shape of the three-dimensionalmodel to an anatomically reasonable shape, an external energy term thatdeforms the three-dimensional model towards one or more characteristicfeature points in the two-dimensional image, and a further energy termthat restricts a deformation of the three-dimensional model to a viewplane associated with the two-dimensional image.

In some embodiments, the method may further comprise processing theadapted three-dimensional model to determine a value for at least oneparameter of the anatomical structure. In some embodiments, the at leastone parameter may comprise any one or more of a volume of at least partof the anatomical structure and a thickness of at least part of theanatomical structure.

According to a second aspect of the invention, there is provided acomputer program product comprising a computer readable medium, thecomputer readable medium having computer readable code embodied therein,the computer readable code being configured such that, on execution by asuitable computer or processor, the computer or processor is caused toperform the method or the methods described above.

According to a third aspect of the invention, there is provided anapparatus for segmenting a two-dimensional image of an anatomicalstructure. The apparatus comprises a processor configured to acquire athree-dimensional model of the anatomical structure, thethree-dimensional model comprising a plurality of segments, and adaptthe acquired three-dimensional model to align the acquiredthree-dimensional model with the two-dimensional image. Thetwo-dimensional image is segmented by the plurality of segments of theadapted three-dimensional model.

In some embodiments, the processor may be configured to control one ormore user interfaces to render the segmented two-dimensional image.

According to the aspects and embodiments described above, thelimitations of existing techniques are addressed. In particular, theabove-described aspects and embodiments allow for the segmentation oftwo-dimensional images using three-dimensional models. By usingthree-dimensional models to segment two-dimensional images, moreinformation can be acquired about the anatomical structure in thetwo-dimensional image (for example, about the shape of the anatomicalstructure out of the plane). The segmented two-dimensional imagesacquired according to the above-described aspects and embodiments can beused for a better visualization of the anatomical structure.Furthermore, the aspects and embodiments can employ many well-testedthree-dimensional models that are already available and these volumetricmodels are not susceptible to variances in the choice of scan plane.Also, employing available three-dimensional models allows the knowledgealready gained from these models to be exploited and saves resources.Therefore, the aspects and embodiments described above enable asuccessful application of three-dimensional models for two-dimensionalimage segmentation.

There is thus provided an improved method and apparatus for segmenting atwo-dimensional image of an anatomical structure, which overcomes theexisting problems.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention, and to show more clearlyhow it may be carried into effect, reference will now be made, by way ofexample only, to the accompanying drawings, in which:

FIG. 1 is a block diagram of an apparatus according to an embodiment;

FIG. 2 is a flow chart illustrating a method according to an embodiment;

FIG. 3 is an illustration of a part of a three-dimensional modelcomprising a plurality of segments according to an embodiment;

FIG. 4A is an illustration of a two-dimensional image according to anembodiment; and

FIG. 4B is an illustration of a two-dimensional image according to anembodiment.

DETAILED DESCRIPTION OF THE EMBODIMENTS

As noted above, the invention provides an improved method and apparatusfor segmenting a two-dimensional image of an anatomical structure, whichovercomes the existing problems.

FIG. 1 shows a block diagram of an apparatus 100 according to anembodiment that can be used for segmenting a two-dimensional (2D) imageof an anatomical structure. The two-dimensional image can be atwo-dimensional medical image such as a two-dimensional computedtomography (CT) image, a two-dimensional magnetic resonance (MR) image,a two-dimensional ultrasound (US) image, a two-dimensional positronemission tomography (PET) image, a two-dimensional single photonemission computed tomography (SPECT) image, a two-dimensional nuclearmedicine image, or any other two-dimensional medical image. Thetwo-dimensional image may be a single slice (or a single plane).

The anatomical structure in the two-dimensional image may be an organsuch as a heart, a lung, an intestine, a kidney, a liver, or any otheranatomical structure. The anatomical structure in the two-dimensionalimage can comprise one or more anatomical parts. For example, a twodimensional image of the heart can comprise a ventricle, an atrium, anaorta, and/or any other part of the heart. Although examples have beenprovided for the type of two-dimensional image and for anatomicalstructure (and the parts of the anatomical structure) in thetwo-dimensional image, it will be understood that the invention may alsobe used for segmenting any other type of two-dimensional image and anyother anatomical structures in the two-dimensional image.

The apparatus 100 comprises a processor 102 that controls the operationof the apparatus 100 and that can implement the method describe herein.The processor 102 can comprise one or more processors, processing units,multi-core processors or modules that are configured or programmed tocontrol the apparatus 100 in the manner described herein. In particularimplementations, the processor 102 can comprise a plurality of softwareand/or hardware modules that are each configured to perform, or are forperforming, individual or multiple steps of the method according toembodiments of the invention.

Briefly, the processor 102 is configured to acquire a three-dimensional(3D) model of the anatomical structure in the two-dimensional (2D) imageand adapt the acquired three-dimensional model to align the acquiredthree-dimensional model with the two-dimensional image. Thethree-dimensional model comprises a plurality of segments and thetwo-dimensional image is segmented by the plurality of segments of theadapted three-dimensional model.

In some embodiments, the apparatus 100 may also comprise at least oneuser interface 104. Alternatively or in addition, at least one userinterface 104 may be external to (i.e. separate to or remote from) theapparatus 100. For example, at least one user interface 104 may be partof another device.

A user interface 104 may be for use in providing a user of the apparatus100 (for example, a healthcare provider, a healthcare specialist, a caregiver, a subject, or any other user) with information resulting from themethod according to the invention. The processor 102 may be configuredto control one or more user interfaces 104 to provide informationresulting from the method according to the invention. For example, theprocessor 102 may be configured to control one or more user interfaces104 to render the segmented two-dimensional image. Alternatively or inaddition, a user interface 104 may be configured to receive a userinput. In other words, a user interface 104 may allow a user of theapparatus 100 to manually enter instructions, data, or information. Theprocessor 102 may be configured to acquire the user input from one ormore user interfaces 104.

A user interface 104 may be any user interface that enables rendering(or output) of information, data or signals to a user of the apparatus100. Alternatively or in addition, a user interface 104 may be any userinterface that enables a user of the apparatus 100 to provide a userinput, interact with and/or control the apparatus 100. For example, theuser interface 104 may comprise one or more switches, one or morebuttons, a keypad, a keyboard, a touch screen or an application (forexample on a tablet or smartphone), a display screen, a graphical userinterface (GUI) or other visual rendering component, one or morespeakers, one or more microphones or any other audio component, one ormore lights, a component for providing tactile feedback (e.g. avibration function), or any other user interface, or combination of userinterfaces.

In some embodiments, the apparatus 100 may also comprise a memory 106configured to store program code that can be executed by the processor102 to perform the method described herein. The memory 106 can also beused to store models, images, information, data, signals andmeasurements acquired or made by the processor 102 of the apparatus 100or from any interfaces, memories or devices that are external to theapparatus 100. For example, the memory 106 may be used to store thetwo-dimensional image of the anatomical structure, one or morethree-dimensional models of anatomical structures (which may, forexample, comprise one or more three-dimensional models for theanatomical structure in the two-dimensional image), the adaptedthree-dimensional model, the segmented two-dimensional image, orsimilar.

In some embodiments, the apparatus 100 may also comprise acommunications interface 108 for enabling the apparatus 100 tocommunicate with any interfaces, memories and devices that are internalor external to the apparatus 100. The communications interface 108 maycommunicate with any interfaces, memories and devices wirelessly or viaa wired connection. For example, in an embodiment where one or more userinterfaces 104 are external to the apparatus 100, the communicationsinterface 108 may communicate with the one or more external userinterfaces wirelessly or via a wired connection. Similarly, in anembodiment where one or more memories are external to the apparatus 100,the communications interface 108 may communicate with the one or moreexternal memories wirelessly or via a wired connection.

It will be appreciated that FIG. 1 only shows the components required toillustrate this aspect of the invention, and in a practicalimplementation the apparatus 100 may comprise additional components tothose shown. For example, the apparatus 100 may comprise a battery orother power supply for powering the apparatus 100 or means forconnecting the apparatus 100 to a mains power supply.

FIG. 2 illustrates a method 200 for segmenting a two-dimensional imageof an anatomical structure according to an embodiment. The illustratedmethod 200 can generally be performed by or under the control of theprocessor 102 of the apparatus 100.

With reference to FIG. 2, at block 202, a three-dimensional model of theanatomical structure in the two-dimensional image is acquired. Thethree-dimensional model of the anatomical structure may comprise one ormore anatomical parts that correspond to anatomical parts in thetwo-dimensional image of the anatomical structure. The three-dimensionalmodel for the anatomical structure may be acquired from the memory 106of the apparatus 100 or from a memory external to the apparatus 100. Forexample, the processor 102 of the apparatus may be configured to acquirethe three-dimensional model for the anatomical structure from the memory106 of the apparatus 100 or from a memory external to the apparatus 100.The three-dimensional model for the anatomical structure may be adeformable three-dimensional model. In some embodiments, thethree-dimensional model for the anatomical structure can be athree-dimensional model that is trained based on volumetric image dataassociated with the anatomical structure (or three-dimensional images ofthe anatomical structure). For example, in some embodiments, the methodmay comprise a training phase in which three-dimensional models of oneor more anatomical structures are learnt.

The three-dimensional model for the anatomical structure comprises aplurality of segments. For example, the three-dimensional model for theanatomical structure can comprise a mesh. Where the three-dimensionalmodel of the anatomical structure comprises a plurality of anatomicalparts, the three-dimensional model may comprise a sub-mesh correspondingto one or more of the plurality of anatomical parts. In someembodiments, the mesh (or sub-mesh) can be a triangular mesh. In otherwords, in some embodiments, the three-dimensional model for theanatomical structure may comprise a plurality of triangular segments. Inembodiments in which the three-dimensional model comprises a sub-meshcorresponding to one or more of the plurality of anatomical parts, eachsub-mesh may comprise a plurality of segments. Although examples areprovided for the form of the segments in the three-dimensional model, itwill be understood that the three-dimensional model for the anatomicalstructure may comprise any other shaped segments (or any other shapedmesh).

In some embodiments, the acquired three-dimensional model may comprise(for example, may define or identify) a view plane associated with thetwo-dimensional image. The view plane may comprise, for example, atwo-dimensional plane through the anatomical structure. For example, theview plane may be a standardised two-dimensional plane through theanatomical structure (e.g. a two-dimensional plane through an anatomicalstructure that is defined with respect to one or more anatomicalfeatures or landmarks associated with the anatomical structure). Thus, aview plane can allow a standardised view of an anatomical structure.This facilitates, for example, better comparison between the anatomicalstructures of different subjects. View planes are defined in manyanatomical imaging applications. For example, in fetal imaging, theabdominal circumference of the fetus is measured in the “abdominalcircumference plane”, which is defined as a plane orthogonal to thehead-to-toe axis, running through the stomach and the upper part of theumbilical vein, whilst not passing through the heart. By defining suchview planes, the variability in biometry measurements is minimized.

In embodiments where the acquired three-dimensional model comprises aview plane associated with the two-dimensional image, the acquiredthree-dimensional model may also comprise information corresponding tothe view plane. For example, the model may comprise information definingthe location of a view plane in the model (for example, the informationmay define the position of the view plane with respect to the model,e.g. the position of the view plane with respect to one or more meshsegments of the model). The view plane that is comprised in thethree-dimensional model may enable rotation and/or translation of thethree-dimensional model, such that the plane intersects the module (or,more specifically, the segments of the three-dimensional model). Thisallows for in-plane rotation and/or translation of the three-dimensionalmodel. The information corresponding to the view plane may therefore beindicative of the location at which the view plane associated with thetwo-dimensional image intersects (or cuts) the acquiredthree-dimensional model (for example, if the two-dimensional image istaken on a particular view plane, information about the same view planemay be comprised in the model). For example, the acquiredthree-dimensional model may comprise information indicative of a spatialposition of the view plane associated with the two-dimensional image inthe three-dimensional model, an orientation of the view plane associatedwith the two-dimensional image in the three-dimensional model, or anyother information, or any combination of information corresponding tothe view plane. The information corresponding to the view planeassociated with the two-dimensional image may be encoded in thethree-dimensional model. In some embodiments, a label may be attached toeach segment of the three-dimensional model that intersects the viewplane associated with the two-dimensional image. In this way, it can beensured that the view plane can be reconstructed by interpolatingcorresponding centre points of the segments (even after adaptation ofthe three-dimensional model). From the information corresponding to theview plane associated with the two-dimensional image, it is possible toestablish a correspondence between the acquired three-dimensional modeland the two-dimensional image to align the acquired three-dimensionalmodel with the two-dimensional image, as will be described later.

Alternatively or in addition to the acquired three-dimensional modelcomprising information corresponding to a view plane, in someembodiments, the acquired three-dimensional model may compriseinformation associated with one or more of the plurality of segments.The information may, for example, correspond to one or morecharacteristic features of the anatomical structure. The one or morecharacteristic features of the anatomical structure may comprisecharacteristic features indicative of one or more boundaries in theanatomical structure. The boundaries may, for example, be indicative ofdifferent parts of the anatomical structure. Examples for the one ormore characteristic features of the anatomical structure include, butare not limited to, an average shape of the anatomical structure,typical shape variations in the anatomical structure, appearanceinformation for the anatomical structure, or any other characteristicfeature, or combination of characteristic features of the anatomicalstructure. In embodiments in which the three-dimensional model for theanatomical structure comprises a mesh, the mesh can be representative ofthe characteristic features of the anatomical structure.

Returning back to FIG. 2, although not illustrated, thethree-dimensional model is initially placed (or positioned) in thetwo-dimensional image. In some embodiments, the initial placement of thethree-dimensional model in the two-dimensional image can comprisedetecting the anatomical structure in the two-dimensional image to placethe three-dimensional model at the location of the anatomical structurein the two-dimensional image. The anatomical structure may be detectedin the two-dimensional image using any suitable feature extractiontechnique (such as the Generalized Hough Transformation, GHT, or anyother feature extraction technique). For the initial placement of thethree-dimensional model in the two-dimensional image, it may be assumedthat the two-dimensional image shows a standardized view of theanatomical structure, such as a specific scan or image plane through theanatomical structure. For example, in an embodiment in which thetwo-dimensional image of the anatomical structure is a two-dimensionalimage of the heart, it may be assumed that the two-dimensional imageshows a standardized view (e.g. a view plane) of the heart (such as atwo chamber view, a three chamber view, a four chamber view, a shortaxis view, a long axis view, or any other standardized cardiac view).

Once the three-dimensional model is placed in the two-dimensional image,the three-dimensional model and the two-dimensional image are broughtinto correspondence. Thus, at block 204 of FIG. 2, the acquiredthree-dimensional model is adapted to align the acquiredthree-dimensional model with the two-dimensional image. For example, theacquired three-dimensional model may be iteratively adapted to align theacquired three-dimensional model with the two-dimensional image.

The adaptation of the acquired three-dimensional model to align with thetwo-dimensional image can comprise rotating the acquiredthree-dimensional model to align the acquired three-dimensional modelwith the two-dimensional image. For example, the acquiredthree-dimensional model may be rotated into the plane of thetwo-dimensional image. Alternatively or in addition, the adaptation ofthe acquired three-dimensional model to align with the two-dimensionalimage can comprise translating the acquired three-dimensional model toalign the acquired three-dimensional model with the two-dimensionalimage. In this way, the three-dimensional model can be reoriented tomatch the orientation of the two-dimensional image. In some embodiments,adapting the acquired three-dimensional model to align with thetwo-dimensional image may comprise restricting the degrees of freedom ofthe acquired three-dimensional model.

In embodiments where the acquired three-dimensional model comprises aview plane associated with the two-dimensional image, the acquiredthree-dimensional model can be adapted to align with the two-dimensionalimage based on the view plane associated with the two-dimensional image.As mentioned earlier, it is possible to establish a correspondencebetween the acquired three-dimensional model and the two-dimensionalimage from the information corresponding to the view plane associatedwith the two-dimensional image to align the acquired three-dimensionalmodel with the two-dimensional image. For example, the informationcorresponding to the view plane associated with the two-dimensionalimage can be used to establish a correspondence between a geometry ofthe acquired three-dimensional model and the two-dimensional image.

The correspondence between the acquired three-dimensional model and thetwo-dimensional image can be used to initialise adaptation of a positionand/or orientation of the model (such as by rotation of the model,translation of the model, or similar) to align the acquiredthree-dimensional model with the two-dimensional image. The acquiredthree-dimensional model may be adapted to align with the two-dimensionalimage based on any one or more of a spatial position of the view plane,an orientation of the view plane, and any other feature associated withthe view plane. In some embodiments, the adaptation of thethree-dimensional model can reduce the degrees of freedom of thethree-dimensional model to one or more planes (such as to in-plane, ormostly in-plane, motion and/or deformation). For example, thethree-dimensional model may first be rotated into the view plane of thetwo-dimensional image according to information relating to anorientation of the view plane that is comprised (or encoded) in thethree-dimensional model. Then, the three-dimensional model may betranslated (for example, in-plane) to the correct position in thetwo-dimensional image using any suitable technique (such as a techniquebased on the Generalized Hough Transformation, GHT). An iterative affineand deformable adaption step may then follow to adapt thethree-dimensional model to the contours in the two-dimensional image,which can be limited to in-plane motion.

In some embodiments, placement of the model in the image mayalternatively or additionally be based on parameters relating to acoordinate system of the subject (for example, the orientation and/orthe position of the subject with respect to the device used to createthe two-dimensional image). For example, in cardiac imaging, atransducer may be placed close to the apex of the heart such that theatria are at the “far side” of the image (with respect to the directionof the travelling sound). Such positional information can be exploitedfor in-plane rotation and/or translation of the three-dimensional modelin order to initially place the three-dimensional model as accurately aspossible.

In embodiments where the acquired three-dimensional model comprisesinformation corresponding to one or more characteristic features of theanatomical structure, the acquired three-dimensional model can beadapted to align with the two-dimensional image based on the informationcorresponding to the one or more characteristic features of theanatomical structure. In some embodiments, for example, adapting theacquired three-dimensional model based on the information correspondingto the one or more characteristic features of the anatomical structurecan comprise adaptation (for example, an iterative adaptation) of thethree-dimensional model into conformity with one or more characteristicfeatures of the anatomical structure in the two-dimensional image. Forexample, one or more characteristic features of the acquiredthree-dimensional model may be matched (or mapped) to one or morecorresponding characteristic features in the two-dimensional model andthe one or more characteristic features of the acquiredthree-dimensional model may then be adapted into conformity with the oneor more corresponding characteristic features of the anatomicalstructure in the two-dimensional image.

In some embodiments, one or more boundaries of the anatomical structuremay be detected in the two-dimensional image. A boundary may, forexample, be detected at a junction between different parts of theanatomical structure in the two-dimensional image. Then, for eachsegment of the acquired three-dimensional model, a target point (forexample, a contour point) may be detected based on one or morecharacteristic features. This target point detection can compriseprojecting a search ray into the plane of the two-dimensional image anddetecting a target point along the projected search ray for each segmentof the acquired three-dimensional model based on one or morecharacteristic features. This detection of target points along theprojected search ray can prevent the technique from searching for targetpoints that lie outside the image plane and thus areas that do notcomprise image information. The one or more characteristic features may,for example, be automatically learned and can comprise, but are notlimited to, grey values and/or edges. In these embodiments, the acquiredthree-dimensional model may be adapted through an attraction of a centreof one or more of the plurality of segments of the acquiredthree-dimensional model to the target point detected for that segment.

In some embodiments, adapting the acquired three-dimensional model maycomprise minimising an energy functional E that attracts thethree-dimensional model to the two-dimensional image to align thethree-dimensional model with the two-dimensional image. The energyfunctional E may comprise an internal energy term E_(int) thatconstrains a shape of the three-dimensional model to an anatomicallyreasonable shape. An anatomically reasonable shape is a shape that lieswithin a range of typical shape variations of the anatomical structurethat is to be segmented and/or that differs within a predefinedtolerance from a mean shape of the anatomical structure that is to besegmented. In other words, the internal energy term E_(int) can ensurethat the adapted three-dimensional model (or the final shape of theadapted three-dimensional model) is reasonable in terms of a knownanatomy for the anatomical structure.

Alternatively or in addition, the energy functional E may comprise anexternal energy term E_(ext) that deforms the three-dimensional modeltowards one or more characteristic feature points in the two-dimensionalimage. For example, in some embodiments, the external energy termE_(ext) may be a term that deforms the three-dimensional model towardsone or more planes (for example, target planes) orthogonal to one ormore image gradients (for example, at the target points mentionedearlier) in the two-dimensional image. The external energy term E_(ext)can stabilise the three-dimensional model such that it does not deviate(or does not deviate strongly or by more than a predefined amount) fromthe one or more planes.

In embodiments in which the energy functional E comprises an internalenergy term E_(int) and an external energy term E_(ext), the energyfunctional may be determined as the sum of the internal energy termE_(int) and an external energy term E_(ext). For example, the energyfunctional E may be expressed as:

E:=E _(int) +E _(ext).

In embodiments in which the three-dimensional model comprises a mesh,the whole mesh may be adapted (or deformed) by minimising the energyfunctional comprising an internal energy term and external energy term.

Alternatively or in addition, the energy functional may comprise afurther energy term that restricts (or constrains) a deformation (oradaptation) of the three-dimensional model to a view plane associatedwith the two-dimensional image. For example, the further energy term mayrestrict deformation of the three-dimensional model to in-plane motionor in-plane deformation. In some embodiments, the restriction (orconstraint) on the adaptation of the three-dimensional model maycomprise attaching one or more springs to one or more segments of thethree-dimensional model. For example, one or more springs may beattached to one or more segments that intersect a view plane associatedwith the two-dimensional image. The addition of springs to one or moresegments of the three-dimensional model can provide a more stableadaptation of the three-dimensional model.

FIG. 3 illustrates a part of a three-dimensional model of an anatomicalstructure comprising a plurality of segments 302 according to an exampleembodiment in which one or more springs 306 are attached to one or moreof the plurality of segments. In this illustrated example embodiment,the three-dimensional model comprises a triangular mesh and thus theplurality of segments 302 comprise a plurality of triangular segments.However, as mentioned earlier, it will be understood that any othershaped segments (or any other shaped meshes) are also possible. Asillustrated in this example embodiment, the springs 306 are attached tothe segments 302 that intersect an image plane 300 of thetwo-dimensional image at an intersection line 304. The springs 306 areattached to the centre points of the intersecting segments 302 torestrict the motion of these segments with respect to the image plane300. For example, the springs 306 can ensure that the intersectingsegments 302 do not move more than a predefined distance from the imageplane 300.

Thus, as described above, the energy functional may comprise any one ormore of an internal energy term E_(int), an external energy termE_(ext), and a further energy term that restricts a deformation (oradaptation) of the three-dimensional model. In an example embodiment inwhich the energy functional E comprises an internal energy term E_(int),an external energy term E_(ext), and a further energy term thatrestricts a deformation (or adaptation) of the three-dimensional model,the energy functional may be determined as the sum of the internalenergy term E_(int), the external energy term E_(ext), and the furtherenergy term that restricts a deformation (or adaptation). In an exampleof such an embodiment, the energy functional E may be expressed as:

E:=E _(int) +E _(ext)+Σ_(j) w _(j)[(n _(plane))^(T)(c _(j) −x _(j))]²

Here, in the further energy term that restricts a deformation (oradaptation) of the three-dimensional model, c_(j) denotes the centrepoints of the plurality of segments and x_(j) denotes the correspondingtarget points in the image plane. In this embodiment, the distancebetween the target points is projected on the normal of the view plane(or the view plane vector), which is denoted by n_(plane), to allow forin-plane motion and penalise (or discourage or prevent) out-of-planemotion. The transpose T of the view plane vector is used to determinethe projection of (c_(j)−x_(j)) on the normal n_(plane) of the viewplane. Optionally, the further energy term that restricts a deformation(or adaptation) can be weighted by a weighting factor w_(j). Theweighting factor w_(j) can be adapted to allow the restriction (orconstraint) to be relaxed or tightened. This can be useful where thereis a high uncertainty in the choice of scan plane during acquisition ofthe two-dimensional image.

Finally, as the acquired three-dimensional model of the anatomicalstructure comprises a plurality of segments, the adaptedthree-dimensional model of the anatomical structure also comprises aplurality of segments. Thus, the two-dimensional image is segmented bythe plurality of segments of the adapted three-dimensional model.

Thus, according to the method described herein, arbitrarytwo-dimensional images (or slices) of anatomical structures can besegmented. Moreover, this is possible without re-training features ofthe three-dimensional model provided that the features relevant to thetwo-dimensional image are encoded in the three-dimensional model.However, in some embodiments, the features (for example, one or moreoptimal features) of a three-dimensional model may be re-trained basedone or more two-dimensional images. For example, this may allow a morespecific adaptation of the three-dimensional model to thetwo-dimensional image of the anatomical structure, which may beparticularly useful if the quality and/or resolution of thetwo-dimensional image varies considerably.

In any of the embodiments described herein, the method may furthercomprise rendering (or outputting or displaying) the segmentedtwo-dimensional image to the user. In other words, the method mayfurther comprise simultaneously rendering (or outputting or displaying)the adapted three-dimensional model comprising the plurality of segmentsand the two-dimensional image. In some embodiments, the processor 102may control one or more user interfaces 104 (such as a display screen ora graphical user interface) to render (or output or display) thesegmented two-dimensional image. In this way, a user can view thesegmented two-dimensional image. This can be useful for medical analysisor assessment. Also, the simultaneous rendering of the adaptedthree-dimensional model comprising the plurality of segments and thetwo-dimensional image allows visualisation of the spatial relationbetween the adapted three-dimensional model and the two-dimensionalimage.

FIG. 4A is an illustration of a segmented two-dimensional imageaccording to an embodiment prior to adaptation of the three-dimensionalmodel to align with the two-dimensional image. In this exampleembodiment, a plurality of anatomical parts 400, 402, 404, 406, 408, 410of an anatomical structure (which in this example is the heart) aredefined by the three-dimensional model. For example, the plurality ofanatomical parts comprise a ventricle, an atrium, an aorta and similar.After placement of the three-dimensional model in the two-dimensionalimage, the three-dimensional model comprising the plurality of segmentsis then adapted to align with the two-dimensional image.

FIG. 4B is an illustration of a segmented two-dimensional imageaccording to an embodiment following adaptation of the three-dimensionalmodel to align with the two-dimensional image. As illustrated in FIG.4B, following the adaptation, the plurality of anatomical parts 400,402, 404, 406, 408, 410 defined by the three-dimensional model arealigned to the corresponding parts of the anatomical structure in thetwo-dimensional image.

It will be understood that the method described herein is not restrictedto a single two-dimensional image. In some embodiments, multipletwo-dimensional images can be combined to achieve a segmentation thatcovers three-dimensional aspects. For example, a plurality of segmentedtwo-dimensional images (such as two orthogonal two-dimensional images orany other plurality of segmented two-dimensional images) may be rendered(or output or displayed) simultaneously. In some such embodiments,cross-plane two-dimensional scans can define contours of the anatomicalstructure in two orthogonal slices and the adapted three-dimensionalmodel can fit both contours and provide a model-based interpolationbeyond the two-dimensional slices.

In any of the embodiments described herein, the method may furthercomprise processing the adapted three-dimensional model to determine avalue for at least one parameter of the anatomical structure. The atleast one parameter may be any parameter associated with the anatomicalstructure. For example, the at least one parameter may comprise any oneor more of a volume of at least part of the anatomical structure, athickness of at least part of the anatomical structure, and any otherparameter of the anatomical structure.

There is therefore provided an improved method and apparatus forsegmenting a two-dimensional image of an anatomical structure. Themethod and apparatus described herein can be used for the segmentationof any arbitrary anatomical structures (for example, organs or any otheranatomical structure) in two-dimensional images. The method andapparatus can be useful in medical imaging analysis and visualisationtools.

There is also provided a computer program product comprising a computerreadable medium, the computer readable medium having computer readablecode embodied therein, the computer readable code being configured suchthat, on execution by a suitable computer or processor, the computer orprocessor is caused to perform the method or methods described herein.Thus, it will be appreciated that the invention also applies to computerprograms, particularly computer programs on or in a carrier, adapted toput the invention into practice. The program may be in the form of asource code, an object code, a code intermediate source and an objectcode such as in a partially compiled form, or in any other form suitablefor use in the implementation of the method according to the invention.

It will also be appreciated that such a program may have many differentarchitectural designs. For example, a program code implementing thefunctionality of the method or system according to the invention may besub-divided into one or more sub-routines. Many different ways ofdistributing the functionality among these sub-routines will be apparentto the skilled person. The sub-routines may be stored together in oneexecutable file to form a self-contained program. Such an executablefile may comprise computer-executable instructions, for example,processor instructions and/or interpreter instructions (e.g. Javainterpreter instructions). Alternatively, one or more or all of thesub-routines may be stored in at least one external library file andlinked with a main program either statically or dynamically, e.g. atrun-time. The main program contains at least one call to at least one ofthe sub-routines. The sub-routines may also comprise function calls toeach other.

An embodiment relating to a computer program product comprisescomputer-executable instructions corresponding to each processing stageof at least one of the methods set forth herein. These instructions maybe sub-divided into sub-routines and/or stored in one or more files thatmay be linked statically or dynamically. Another embodiment relating toa computer program product comprises computer-executable instructionscorresponding to each means of at least one of the systems and/orproducts set forth herein. These instructions may be sub-divided intosub-routines and/or stored in one or more files that may be linkedstatically or dynamically.

The carrier of a computer program may be any entity or device capable ofcarrying the program. For example, the carrier may include a datastorage, such as a ROM, for example, a CD ROM or a semiconductor ROM, ora magnetic recording medium, for example, a hard disk. Furthermore, thecarrier may be a transmissible carrier such as an electric or opticalsignal, which may be conveyed via electric or optical cable or by radioor other means. When the program is embodied in such a signal, thecarrier may be constituted by such a cable or other device or means.Alternatively, the carrier may be an integrated circuit in which theprogram is embedded, the integrated circuit being adapted to perform, orused in the performance of, the relevant method.

Variations to the disclosed embodiments can be understood and effectedby those skilled in the art in practicing the claimed invention, from astudy of the drawings, the disclosure and the appended claims. In theclaims, the word “comprising” does not exclude other elements or steps,and the indefinite article “a” or “an” does not exclude a plurality. Asingle processor or other unit may fulfil the functions of several itemsrecited in the claims. The mere fact that certain measures are recitedin mutually different dependent claims does not indicate that acombination of these measures cannot be used to advantage. A computerprogram may be stored/distributed on a suitable medium, such as anoptical storage medium or a solid-state medium supplied together with oras part of other hardware, but may also be distributed in other forms,such as via the Internet or other wired or wireless telecommunicationsystems. Any reference signs in the claims should not be construed aslimiting the scope.

1. A method for segmenting a two-dimensional image of an anatomicalstructure, the method comprising: acquiring a three-dimensional model ofthe anatomical structure, the three-dimensional model comprising aplurality of segments and a view plane associated with thetwo-dimensional image; and adapting the acquired three-dimensional modelto align the acquired three-dimensional model with the two-dimensionalimage based on the view plane associated with the two-dimensional image;and wherein the two-dimensional image is segmented by the plurality ofsegments of the adapted three-dimensional model.
 2. The method asclaimed in claim 1, wherein the view plane associated with thetwo-dimensional image comprises a two-dimensional plane through theanatomical structure.
 3. The method as claimed in claim 2, wherein thetwo-dimensional plane through the anatomical structure is defined withrespect to one or more anatomical features associated with theanatomical structure.
 4. The method as claimed in claim 1, wherein theacquired three-dimensional model is adapted to align with thetwo-dimensional image based on any one or more of a spatial position ofthe view plane and an orientation of the view plane.
 5. The method asclaimed in claim 1, wherein the acquired three-dimensional modelcomprises information associated with one or more of the plurality ofsegments, the information corresponding to one or more characteristicfeatures of the anatomical structure.
 6. The method as claimed in claim5, wherein the acquired three-dimensional model is adapted to align withthe two-dimensional image based on the information corresponding to theone or more characteristic features of the anatomical structure.
 7. Themethod as claimed in claim 1, wherein adapting the acquiredthree-dimensional model to align with the two-dimensional imagecomprises any one or more of: rotating the acquired three-dimensionalmodel to align the acquired three-dimensional model with thetwo-dimensional image; and translating the acquired three-dimensionalmodel to align the acquired three-dimensional model with thetwo-dimensional image.
 8. The method as claimed in claim 1, whereinadapting the acquired three-dimensional model to align with thetwo-dimensional image comprises restricting the degrees of freedom ofthe acquired three-dimensional model.
 9. The method as claimed in claim1, wherein adapting the acquired three-dimensional model comprisesminimizing an energy functional that attracts the three-dimensionalmodel to the two-dimensional image to align the three-dimensional modelwith the two-dimensional image
 10. The method as claimed in claim 9,wherein the energy functional comprises any one or more of: an internalenergy term that constrains a shape of the three-dimensional model to ananatomically reasonable shape; an external energy term that deforms thethree-dimensional model towards one or more characteristic featurepoints in the two-dimensional image; and a further energy term thatrestricts a deformation of the three-dimensional model to a view planeassociated with the two-dimensional image.
 11. The method as claimed inclaim 1, the method further comprising: processing the adaptedthree-dimensional model to determine a value for at least one parameterof the anatomical structure.
 12. The method as claimed in claim 11,wherein the at least one parameter comprises any one or more of a volumeof at least part of the anatomical structure and a thickness of at leastpart of the anatomical structure.
 13. A non-transitory computer programproduct comprising a computer readable medium, the computer readablemedium having computer readable code embodied therein, the computerreadable code being configured such that, on execution by a suitablecomputer or processor, the computer or processor is caused to performthe method of claim
 1. 14. An apparatus for segmenting a two-dimensionalimage of an anatomical structure, the apparatus comprising: a processorconfigured to: acquire a three-dimensional model of the anatomicalstructure, the three-dimensional model comprising a plurality ofsegments and a view plane associated with the two-dimensional image; andadapt the acquired three-dimensional model to align the acquiredthree-dimensional model with the two-dimensional image based on the viewplane associated with the two-dimensional image; wherein thetwo-dimensional image is segmented by the plurality of segments of theadapted three-dimensional model.
 15. The apparatus as claimed in claim14, wherein the processor is configured to control one or more userinterfaces to render the segmented two-dimensional image.